1 METHOD/APPARATUS FOR 

2 IMPROVING WEB (INTERNET) ACCESS 

3 

4 Cross Reference To Related Applications 

5 This application claims the benefit of U.S. Provisional Application Serial 

6 No. 60/174,677, filed January 6, 2000, and U.S. Provisional Application Serial No. 

7 60/181,260, filed February 9, 2000. 

8 Field of the Invention 

9 The present invention relates to capturing events (time, audio, and/ or 



10 venue) or location (gps position, still video) into a device memory as a selected 

11 placemark, which placemark then operates to extract links to additional 

12 information via the world-wide web. 
13 



14 Background 

15 The internet has recently gained tremendous popularity. Consumers are 

16 unable to turn on the radio or read the paper without learning about some new 

17 internet phenomenon or the rapid rise of e-commerce. However, the internet is 

18 still a place separate from the world that we actually live in. Most of the 

19 experiences consumers have during the day are largely devoid of internet 

20 content, and when individuals want to "get on" the internet, they are forced to 

21 chain themselves to a seat in front of a monitor in order to "enter" the vast world 

22 of network information. 



Stated another way, internet providers are focusing more and more on 
continuous, wireless and universal access to the internet. Internet based 
businesses are trying to improve their web based products and services to 
motivate the on-line consumer to eventually spend money, either directly or 
indirectly, through their URL, However, the 2 dimensional flat screen world of 
the internet remains largely divorced from the 3 dimensional world of 
experiences and interactions that consumers travel during 99% of a working day. 
Consequently, most consumer time is spent navigating the labyrinth of the 
internet instead of obtaining the specific information consumers desire. 

How well consumers can make use of the internet is now primarily based 
on how well one can find their way through the labyrinth of URL's, and screen 
after screen of links and subsets. Web browsers therefore all have a sense that 
what they are looking for is out there, but few really know how to get there 
efficiently, if at all. What's more, almost all the things about which browsers 
may be most interested are presented in a format that is not internet based. It's 
an article, a product, a television show, a book, a radio program or something 
else that becomes part of one's everyday experience, which prompts individuals 
to want to learn more about a topic or issue. 

Every time consumers look at something or hear about something in the 
"real world" there is likely to be a vast amount of interesting and applicable 
information about it somewhere in the "web" world. Whoever or whatever was 



1 involved in creating the things that are seen, the items that one holds, or the 

2 words read or transmitted, has much more to tell us, but can't practically reach 

3 us because of space, time or financial constraints. In addition, since the 

4 information supplier is unaware whether any particular individual wants more 

5 than top level information, there is always a tradeoff about how much 

6 information to initially provide. 

7 Ideally, when individuals have an experience with something during the 

8 day, they should be able to get information about it without having to guess 

9 where it is or how to get it. Starting to look for information at the beginning, or 

10 even middle of the "web" labyrinth, is often frustrating and unnecessary. 

11 Therefore, creating a means for linking the real world to the world of electronic 

12 information is essential to improving the consumers 7 incessant desire to 

13 maximize their day-to-day learning experience. 

14 With regards to the foregoing, a number of disclosures have recently come 

15 forward aimed at internet access. For example, U.S. Patent No. 5,832,223, 

16 entitled "System, Method and Device for Automatic Capture of Internet Access 

17 Information In A Broadcast Signal for Use by an Internet Access Device" relates 

18 to automatic capture of internet access information in a broadcast signal for use 

19 by an internet access unit. The system described therein first comprises a 

20 broadcast unit having an internet access information unit, coupled to receive 

21 internet access information, for encoding the internet access information into a 
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1 broadcast signal to provide an augmented signal and broadcasting said 

2 augmented signal. The system also is said to include a receiving unit for 

3 receiving the augmented signal for a user, a capture unit coupled to the receiving 

4 unit and interfaced with the internet, for automatically storing/ utilizing the 

5 internet access information in accordance with a predetermined scheme at a 

6 predetermined time upon intitiation of the predetermined scheme. In addition, a 

7 display/ television screen/ audio unit is coupled to the access unit and, where 

8 selected, to the receiving unit, for displaying either the video/ audio/ textural 

9 information obtained by using the internet access information, or video, audio, 

10 textural information from the broadcast signal. 

11 Attention is also directed to U.S. Patent No. 5,961,603, entitled "Access 

12 System and Method for Providing Interactive Access to an Information Source 

13 Through a Networked Distribution System". This disclosure relates to a system 

14 and method for accessing internet based information through a user television in 

15 a television distribution network. It is said to enable a user to access and view 

16 information which is related to the programming content of a currently viewed 

17 television broadcast. The concept is defined as "channel hyperlinking", and is 

18 said to enable a television viewer to dowload internet web pages and the like, 

19 which are related to a currently viewed advertisement, program, newscast, etc. 
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1 Accordingly, it is a general object of the invention to provide both an 

2 improved product and process to allow consumers a more efficient and 

3 improved pathway / link to additional information via the world-wide web. 

4 Along such lines, it is a more specific object of this invention to provide 



5 consumers with an improved product and process to placemark daily events into 

6 a device memory, and later rely upon such placemarking to automatically and 

7 conveniently provide the consumer corresponding links on the world-wide web. 

8 In addition, it is also an object of this invention to recognize that the 

9 aforementioned events, as well as physical location itself, can and will prompt a 

10 consumer to seek additional information. Accordingly, the invention herein 

11 thereby uniquely identifies that such factors (events and/or location, etc.,) can 

12 now be comprehensively identified and automatically associated with a 

13 corresponding web link for convenient delivery of supporting information 

14 relating to such event and / or location. 

15 Finally, it is an object of this invention to provide a product and process 

16 for web searching and information gathering that is more closely tied to the way 

17 that consumers collect and sort information, via a consumer's senses (touch, 

18 vision, hearing) in relation to time. 

19 Summary of the Invention 

20 A product and process is disclosed, which captures an event (time, audio 

21 and/ or venue) or location (gps position, still video) into memory as a selected 
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1 placemark, which placemark then readily and conveniently extracts links to 

2 additional information via the world-wide web. The product/ process is 

3 uniquely applicable to the placemarking of events associated with such 

4 consumer media as television and/ or radio, and further includes the 

5 placemarking of written information supplied via printed media. In addition, a 

6 process will be described where consumers can also extract information from the 

7 internet based on their physical location, e.g., in front of a building, in the middle 

8 of a town, by a billboard, etc. 

9 In alternative embodiment, a system is disclosed for capturing internet 

10 access information related to the following variables, or at least two of the 

11 variables in combination; (1) video aspects; (2) audio aspects; (3) geographical 

12 position or movement; and (4) time, comprising: 



13 (a) a capture unit comprising: 

14 (i) a database for receiving and storing at least two or more of 

15 said variables; 

16 (ii) a selector which upon actuation thereof stores at least two or 

17 more of said variables; 

18 (b) an interface for coupling said capture unit to said internet to 



19 access internet information wherein said interface extracts links from an internet 

20 web site which supplies said information related to two or more of said variables 

21 stored in said capture unit database. 
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1 Brief Description of the Drawings 

2 For a better understanding of the present invention, together with other 

3 objects / features and advantages, reference should be made to the following 

4 description which should be read in conjunction with the following figures 

5 wherein like numerals represent like parts: 

6 FIG, 1 is a schematic illustration of an exemplary computer system useful 

7 in connection with embodiments of the one or more invention; 

8 FIG, 1A is a schematic illustration of an exemplary television-based 

9 system consistent with the invention; 

10 FIG, 2 is a schematic illustration of an exemplary radio-based system 

11 consistent with the invention; 

12 FIG, 3 is a schematic illustration of an exemplary printed matter based 

13 system consistent with the invention; 

14 FIG, 4 is a block diagram showing an exemplary process flow consistent 

15 with the invention; and 

16 FIG. 5 is a block diagram showing another exemplary process flow 

17 consistent with the invention. 

18 FIG. 6 is a more detailed block diagram, further illustrating the process 

19 flow of the present invention. 
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1 Detailed Description of Preferred Embodiments 

2 The invention described here is a combination of products and systems 

3 (methods) that allow a consumer to more effectively link their world to the 

4 world of information - the web. Any time a consumer wants more information 

5 about anything, something in the real world "triggered" that desire. This 

6 disclosure will focus on how consumers can identify and store these triggers, 

7 which can then be turned into opportunities for more information through the 

8 internet. 

9 There are two primary areas around which this web information system is 

10 defined, time (event) based triggers and space based triggers (a physical item or 

11 location). For the time based arena, the two areas which are most commonly 

12 known to consumers are television or radio. Of course there axe also other event- 

13 related triggers (like a conversation with another person), which will also be 

14 discussed herein. The space based web information system makes a distinction 

15 between two basic information triggers, those that are largely stationary and 

16 those that are mobile. Stationary items would include buildings, real estate, 

17 towns, counties, states, billboards... etc. Mobile items would include, but are not 

18 limited to physical products, printed material (magazines, newspapers, books, 

19 coupons... etc), and even items such as vehicles. A consumer's experience with 

20 any of these often triggers a desire for more information, and the present 



8 



1 invention uniquely provides a more efficient and novel technique for the 

2 consumer to access this information. 

3 1. Time Based Trigger - Television 

4 According to the present invention, while a viewer is watching television, 

5 each new screen image, advertisement or topic can provide a link for the viewer 

6 regarding where and how to find more information. Anytime an individual now 

7 views something interesting, they can be conveniently delivered to a specific 

8 internet address that provides a location to find more information - i.e., 

9 preferably not a home page, which is just the middle of the labyrinth. 

10 That is, in the context of the present invention, the specific internet 

11 address delivered to the consumer will satisfy the consumer's curiosity for 

12 additional information, provided by their viewing experience, and does not 

13 simply offer another generic form of the information digested by the consumer, 

14 which information prompted the inquiry. 

15 Furthermore, most individuals do not want to interrupt a program (or 

16 television viewing time, in general) to obtain more information about an 

17 advertisement or topic. Ideally, individuals like to investigate the topic upon 

18 completion of the program. In accordance with the present invention, such 

19 option is now realized. 

20 Of course, it is always possible for a network to provide a URL anytime 

21 something is shown or discussed on TV. However, practically speaking, this is 

9 



1 visually annoying and typically impractical, since the URL's would be painfully 

2 long to contain any degree of specificity. In addition, individuals are usually 

3 unable to write them down in real time, thereby making such option largely 

4 unrealistic. Furthermore, when URL's are shown on TV (usually for ads), they 

5 are ordinarily for a general home page, and not a particular topic or piece of 

6 information. 

7 In such context, the present invention provides a method for 

8 automatically tracking and storing what topics people find interesting and 

9 allowing them to access information about them at their convenience. Of course, 

10 with the advent of webTV™, there is currently a vehicle for accessing the 

11 internet while watching television. However, it will be a long time before the 

12 majority of Americans are connected to interactive webTV, and furthermore, in 

13 webTV, there is no opportunity for allowing users to unobtrusively track 

14 interesting topics for subsequent viewing. 

15 Accordingly, in an exemplary embodiment of the invention as shown in 

16 FIGS. 1 and 1A, a user is provided with a remote control 10 for their television 

17 12. A button 14 on this remote can be depressed anytime a user finds something 

18 of interest, e.g. in a television program during an advertisement, or in the middle 

19 of a show. The network (and/ or advertiser) can then provide as much 

20 information about a TV moment as deemed appropriate. 
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1 For example, if someone were watching a sitcom and saw a product in a 

2 scene about which they wanted more information, they could depress the button 

3 14. By depressing button 14 at such moment the viewer may capture links to: 

4 the sitcom's home page, the network's home page, an information/ order 

5 subpage from the manufacturer of the product, a place to buy a CD for the 

6 background music . . . etc. Alternatively, someone pushes the button 14 during 

7 an ad, they could obtain an information/ order subpage from the manufacturer 

8 of the product (or supplier). Expanding on this concept, if someone depresses 

9 button 14 during a nature show, in the middle of a segment about, e.g., gorillas, 

10 they may obtain links to specific gorilla studies, information about the 

11 videographers and photographers involved in creating the show, maps about 

12 concentrations of the gorilla population, information about the countries in 

13 which gorilla's primarily reside . . . etc. Accordingly, regardless of which 

14 viewing experience provokes the viewer to press button 14, it will always be up 

15 to the television programmer to decide what is relevant and what should 

16 ultimately be made available to the consumer to satisfy their quest for additional 

17 information 

18 One exemplary technique of capturing information consistent with the 

19 invention, considers the combination of time (of the day) and the channel 

20 (network) that the viewer is watching. In such scenario there is no need for any 

21 alteration of existing televisions or signals to capture the information. The 

11 



1 remote 10 has a real time clock (so it "knows" the time) and also knows which 

2 channel is "on", as it was the device that tuned the TV to a particular station. (In 

3 addition, the device can also store it's location (Boston, Philadelphia... etc) if it 

4 has either a gps or a means for programming or storing this information.) 

5 With reference also to FIG. 4, which is an exemplary block diagram of the 

6 process of the present invention, when the viewer depresses the button 14, they 

7 are simply putting a placemark in time for the channel they are viewing. 

8 Accordingly, as shown at 16, there is capture of the desired placemark in 

9 memory on the remote, followed 18, extraction of links from a dedicated site or 

10 central site, and 20, viewing of the links for desired information. In that regard, 

11 and as more fully discussed herein, the presently disclosed device may also serve 

12 as an interactive device, in the sense that the device collects information about 

13 viewing habits of an individual. The device can therefore, when extracting links, 

14 elect to download such information to a ratings service that collects and 

15 distributes information about consumer television viewing trends. 

16 When the user then logs onto the web, they are therefore taken directly to 

17 a website (the "placemark" web site) specifically created for the purpose of 

18 interpreting information downloaded from the web information device. In a 

19 preferred embodiment, the device would have stored information which would 

20 direct their web interface device 22 (computer, PDA... etc) to the "placemark" 

21 web site directly. In one embodiment, the device would not only direct the user 

12 



1 to the "placemark" site, but would also be responsible for logging the user on to 

2 the internet. Once the remote (or other web info device) was "docked" and the 

3 "placemark 7 ' web site reached, placemarks would be downloaded from the 

4 remote (or other web info device) to the site. The centralized web site would 

5 then extract the links associated with a given network at a specific point in time 

6 from a database provided by the venue provider. The site would then 

7 subsequently display pertinent web links for each trigger (event) to the user. In 

8 one preferred embodiment, the information would be displayed to users in a 

9 way that would show a chronology of all triggers which created their 

10 corresponding placemarks, including such items as venue (TV station), program, 

11 time, audio clip, still video, brief summaries and/ or web links. The user could 

12 then click on a title or icon and be brought to the specific location of the 

13 information for viewing 20. They may also need to specify in what area and/ or 

14 time zone they were watching to get the accurate info (if this is not pre-specified 

15 by the device or software). If necessary, it is possible to have a synchronization 

16 signal issued at the beginning of a show for the remote to make certain that the 

17 absolute time is generally correlated to a clock in the remote. Of course, if the 

18 remote is more sophisticated, like a PDA (Palm), it could access the internet 

19 directly at anytime and find pertinent information without a dedicated PC. 

20 With reference to FIG. 5, another exemplary method of capturing 

21 information involves communicating the link information via a signal which is 

13 



1 carried on top of the normal picture and audio television signal. Alternatively, 

2 the signal may carried within the normal picture and/ or audio television signal 

3 This signal can be stored by the TV 12 or remote 10 (communicated by either a 

4 docking with the TV or through wireless R.R) when the user depresses the 

5 button 14. These links can then be explored through docking the remote 10 with 

6 a PC interface 22 (as before) or directly through the television (webTV or 

7 sophisticated remote). It is also possible to download this information directly to 

8 a box connected to a PC through either an R.F. or a signal carried on top of AC 



9 power. 

10 Pursuant to the present invention, the network and its advertisers have 

11 good reason to provide the invention herein as it offers: 

12 • Better, more integrated service to customers. 

13 • More opportunities for generating revenue through advertised or 

14 ancillary items. 

15 • Information about who is viewing the shows and accessing the extra 

16 information. 

17 2. Time Based Trigger - Radio 

18 In a further embodiment of the present invention, providing information 

19 about topics someone hears on the radio can have much (or more) value than it 

20 does for television. Unlike television, where someone could theoretically run to 

21 their computer (or execute a webTV link), most radio listeners are in a remote 
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1 location - a car, the beach, the yard . . . etc. The confusion over the spelling of a 

2 name alone is enough to warrant implementation of the present invention. 

3 Indeed, in spite of diligent efforts by announcers to somehow communicate the 

4 spelling of a word or name, people ordinarily have difficulty remembering. 

5 Listeners typically scramble for a pencil to try and capture the information before 

6 they forget it. According to the present invention, there is now an opportunity 

7 to obtain more information about any topic related to what a listener is now 

8 hearing . . music, talk radio, sports, news, . . . etc. 

9 With reference to FIG. 2, the system herein operates in a manner similar to 

10 the TV system, with a remote control tracking the station and time when a user 

11 depresses the indicated button. However, in most cases, there is no remote for a 

12 radio. Where there is no remote, either for a radio or television system, a 

13 separate button can be provided directly on the radio or television to capture the 

14 time and tuned station when the button is depressed. They would be 

15 downloaded by either wire, R.R or I.R. to a transport device — very much like a 

16 remote (or PDA). This remote is then placed into a docking station and 

17 information is made available as described above with respect to television. 

18 Alternatively, with respect to the radio, the AM and FM signals could also 

19 have linking information carried on their primary audio signals, or within said 

20 signals. When a user depresses the indicated button (Fig. 2), the addresses of 
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1 relevant information would be stored in the radio and then subsequently 

2 downloaded to a transport device by IR, RF or a physical connection. 

3 Various modifications and alternative embodiments are possible. For 

4 example, if a remote can "read" the station a radio is currently tuned to, then the 

5 system can operate exactly like the television remote with no alterations to 

6 existing signals. This can be done through I.R., R.F., wire or with the aid of a 

7 visual scanner, which will actually recognize the display on a radio. As 

8 mentioned before, if the remote is sophisticated enough (like a PDA), it can 

9 access the information directly through the web. 

10 In addition to the above, those skilled in the art will recognize that, e.g., in 

11 the case of an audio broadcast, such may comprise audio produced from a device 

12 that plays prerecorded audio, such as a compact disc (CD) or tape player. In like 

13 manner, a video broadcast herein may simply comprise a video signal produced 

14 from playback of a video cassette recorder (VCR) or digital video disc (DVD). 

15 Furthermore, in the broad scope of the present invention, audio herein may also 

16 comprise speech, in which case the system herein for capturing internet 

17 information would recognize such speech as providing an internet address and 

18 access such internet address when coupling to the internet. 

19 3. Mobile Space Based Triggers - Printed Material/Physical Items 

20 Another group of items that stimulate consumers to desire more 

21 information is printed material (newspapers, magazines, advertisements, flyers, 

16 



1 labels, business cards . . .etc.) This group also includes 3 dimensional physical 

2 items because the information is relayed through the use of contact printing or 

3 labels. The purpose of using the invention herein for such objects is the same as 

4 it is for television or radio - connecting people directly with pertinent 

5 information. 

6 With respect to FIG, 3, this system involves the use of an encrypted icon 

7 30 or indicia that is placed in proximity to the printed material and can be 

8 scanned by the remote 10. The icon is simply printed next to any item for which 

9 there may be a reason to connect users to more information about a topic - a 

10 newspaper article, an ad, a label on a product , advertisement on the side of a 

11 truck. . . etc. In its simplest form, this could be a bar code. If more information is 

12 desired, it could be a 2-D indicia (post office new type) or a custom pattern. The 

13 icon may provide several different types of information. For example, it may 

14 provide a unique serial number, which is a triggered to download all appropriate 

15 links when someone connects to the site, or it may provide the URL's directly 

16 without needing to go through the site. In either case, a user would simply 

17 point the PDA at the indicia and collect the information. A consumer can, 

18 therefore, walk up to just about anything, scan an icon and have an easy method 

19 to secure additional information. For a product, it could be use or maintenance 

20 instructions, major distributors, current or related models, places to buy on-line, 

21 service locations, features, recent favorable reviews . . . etc. And, of course, all of 

17 



1 this enables customers to purchase items and provide information to URL 

2 owners about who is hitting their site and where they obtained the information 

3 (as described before, all encrypted in the indicia). And, like the time-based 

4 scenarios, the providers will have the ability to continuously "adjust" the 

5 links/ information associated with a given indicia as time passes. For instance, 

6 the indicia on a book which was recently reviewed by the New York Times, can 

7 now direct a user to the recent review without changing the indicia - just the 

8 software database. 

9 Because the cost of digital imaging will continue to decrease and the 

10 resolution will continue to increase, the system will eventually not use dedicated 

11 device scanners, but built in digital cameras with software capable of recognizing 

12 and " pulling out" indicias from digital images for processing. As described later, 

13 since imaging will be part of the user's chronology, this will enable someone to 

14 capture more than a numerical association with an abstract barcode. The user 

15 will also have a version of the image they actually saw when an item "triggered" 

16 their interest. 

17 Quite apart from the foregoing, in yet a further embodiment of the present 

18 invention, in those situations wherein the product/ process herein is applied, e.g., 

19 to a three-dimensional object, or any object of interest to the user, the 

20 requirement of an encrypted icon or similar indicia on the object can be rendered 

21 completely unnecessary, in favor of the use of the general visual (or audio) 
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1 aspects/ features of the object at issue. In that regard, the use of general visual 

2 (or even audio) aspects, as used herein, is reference to the fact that any object will 

3 itself, on its own, serve as sufficient visual (or audio) indicia for the device herein 

4 to placemark in memory, which placemarked imaged (or audio) can then be 

5 matched with a stored database of general visual (or audio) aspects, when linked 

6 to the world-wide web, followed by delivery of the user to the appropriate URL 

7 for the object in question. 

8 For example, as applied to a given consumer product, it is typical for such 

9 products to contain within its general visual image an indicia such as an insignia, 

10 logo, trademark, or other visible feature, which can be captured as part of the 

11 general visual image, and properly matched with a database of such indicia 

12 when connecting to the web. Alternatively, the use of an insignia, logo or 

13 trademark can be replaced completely, in the sense that a consumer object itself 

14 will, as noted, have sufficient and distinct general imaging (or audio) 

15 characteristics, or what one can understand as a general and visual (or audio) 

16 fingerprinting characteristic, which can be stored in device memory, and when 

17 compared to a database of such general visual (or audio) fingerprinting 

18 characteristics, serve to bring the user to the appropriate web site. Furthermore, 

19 in the case of audio extraction, the audio may be such audio that the listener 

20 does not necessarily hear, providing an "invisible" audio indicia for the device 

21 herein. For example, subtones or overtones would be one preferred example. 

19 



1 Expanding further on the use of a general visual (or audio) fingerprinting 

2 characteristic, the device herein also contemplates the use of optical character 

3 recognition (OCR), in the sense that, e.g., when a written document is place- 

4 marked, written information is detected (OCR), and such written information 

5 itself can serve as the identifier for directing the user to a desired web site. In 

6 terms of such example, the device herein could be used, for instance, to scan a 

7 document containing a foreign language of which the user is unfamiliar. By then 

8 downloading such foreign language image to the internet, the device herein 

9 would then couple the user to, e.g., a translation internet location, or other 

10 similar site, so that the user could conveniently translate the information 

11 previously presented by the "real world". 

12 Of course, in those situations wherein the device herein captures an image 

13 (or audio) containing more than one visual (or audio) fingerprinting 

14 characteristic, and each visual (or audio) fingerprint itself will therefore have its 

15 own URL, and the user will, accordingly, be supplied with a list of possible 

16 internet sites for which the user can then conveniently select those which he/she 

17 would like to visit in order to obtain additional information regarding the object 

18 in question. 

19 In that regard, the system herein contemplates that on the internet side of 

20 the invention, a complete database of visual (or audio) fingerprints will be 

21 created and stored into memory, which memory database will then serve as the 
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1 appropriate comparative database for which the device herein will identify and 

2 select the appropriate match, and provide the user with the appropriate web site 

3 link for the purpose, once again, of obtaining additional information regarding 

4 the object in question. That being the case, the device herein uniquely delivers 

5 the consumer to a URL, by simply capturing a general visual (or audio) 

6 fingerprinting characteristic of the object of interest, thereby rendering it 

7 unnecessary to use present day features such as bar-coding which must be 

8 separately created and installed on a device, for electronic identification thereof, 

9 4. Stationary Space Based Triggers - Real Estate, Towns, 

10 Buildings . . . etc. 

11 Another group of items which can trigger a consumer's desire for more 

12 information can be organized under the "Stationary Space Based" category. 

13 Anytime someone is anywhere, they will be able to get "relevant" information 

14 for that particular location by hitting a button. This button will store the user's 

15 physical location by committing a gps signal to memory. Like the system 

16 described for television, radio and mobile items, the web info device containing 

17 the gps will be "docked" into a web accessible device (if it does not have 

18 embedded web access). The set of coordinates will be downloaded to the 

19 "placemark" web site, where they will be compared to a database. The software 

20 will then extract from the database the appropriate information which 

21 corresponds to the coordinates. The information will then be presented to the 
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1 user in a form that will enable them to access related links directly. In effect, the 

2 business model is based on the sale of "virtual real estate" to advertisers or 

3 retailers who provide the relevant links to users through the database. Any 

4 particular location may have one or more advertisers based on what is 

5 happening at that location. Here are some examples: 



6 m Clicking the button on a quiet suburban residential street may take 

7 a user to the home page for the town they are in. 

8 • Clicking in front of a house for sale on another residential street 

9 may take a user to the town's home page, the sub-page for that 

10 home's listing under the realtor's site, containing specifications, 

11 price, status and contact information. 

12 • Clicking at a train station west of Boston may take someone to the 

13 MBTA's home page, the sub-page for the western commuter rail 

14 schedule, status of delays or important news. 

15 * Clicking in the middle of the New York thruway can tell someone 

16 about road status (weather conditions, delays. . .etc), places to eat or 

17 refuel at the next exit... etc. A link could take someone directly to 

18 the menu of a little diner or sub shop in the center of a town that is 

19 accessed by the next exit. Another could provide information 

20 about important historical landmarks or points of interest. 
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1 • Clicking in the middle of a National Park could take someone to 

2 information about nearby trails, how to signal to rangers in case of 

3 trouble... etc. 

4 • Clicking on a busy road could take someone to the sites of roadside 

5 stores, or list links to subpages about employment opportunities at 

6 nearby locations. 

7 Of course, the possibilities are endless and it is not realistic to list them all 

8 in this disclosure. Other enhancements to the system include: 

9 $ Capturing altitude information to isolate businesses on multi-story 

10 dwellings 

11 • Capturing the movement of users: direction, speed... etc to figure 

12 out what information should be provided, e.g. east bound exit 

13 information vs. west bound exit information. 

14 ♦ Identifying the orientation of the capture device to provide even 

15 better specificity about what the user may be "pointing" at. This 

16 could be very valuable in busy areas where many things could 

17 trigger a user's interest. This could be "line of site" determination 

18 or based on triangulation. 

19 • Comparing images captured during the button press to stored 

20 images in the database in a general vicinity to make a better 
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1 determination about the users inquiry and provide better 

2 information. 

3 Having emitters (and potentially receivers) in the space based triggers 

4 (stationary or mobile) which can "lock on" to a user's gps device 

5 when they depress their button and download a specific 

6 coordinate, or even list of links through RF. 

7 Integrated System Description 

8 Although each of the scenarios above operate independently of the others, 



9 the invention herein also can integrate the functions required for time based and 

10 space based material into a single remote device. Because more and more 

11 consumers are using items like a Palm™ PDA for all sorts of daily functions, a 

12 remote in accordance with the present invention may be readily configured to be 

13 "Palm-like" or at least PDA-like in nature. People have already started to justify 

14 carrying around electronic assistants for their daily responsibilities and there is 

15 no reason to require them to carry around yet another device. In addition, 

16 programs have already been created to make PDA devices act like television 

17 remotes, and Symbol Technologies has already included scanning capabilities in 

18 PDA devices. 

19 An integrated system will be able to provide a historical chronology of all 

20 things that either piqued a user's curiosity for more information or were of 

21 enough interest or importance to warrant remembering. Many of the 
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1 chronologued items can be linked to web information through the previously 

2 described centralized web site databases, but not necessarily all of them. A 

3 meeting with a person, for example, is unlikely to have web information 

4 associated with it (unless they provide a business card with a web indicia). In 

5 this case, it may be the person's face or some things they were saying which were 

6 worth remembering. The ubiquitous questions, "Where did my week go?" and 

7 "Where did my day go?" can now be answered to any degree specificity that a 

8 user wishes to capture it. When a user docks their capture device, which 

9 includes some combination of storing the time, venue, location, audio and image 

10 associated with a specific button click, they are downloading an electronic diary 

11 of where their time was spent and what caught their interest. In addition, this 

12 can be extended to the concept of displaying the chronology of a user's telephone 

13 calls with audio capture, or a particular business transaction. Some of these 

14 items will have direct web information associated with them and others will not. 

15 In addition to providing relevant web information by making the user their 

16 "own search engine", this can also provide value to individuals who need/want 

17 to reflect back on their time as a part of their jobs - lawyers, doctors, consultants 

18 of any kind. . . etc. 

19 A user will be able to edit their history, deleting unimportant information 

20 and adding their own notes about a particular circumstance. In effect, the tool 

21 can be used as a reverse "Outlook" or other schedule planning software with 
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1 much more detailed information - much of it with web based relevant 

2 information. 

3 As described before, time based triggers with specific venues, relevant 

4 audio or image clips can be supplied by the venue provider and downloaded to 

5 the user when they "dock" their capture device. Additionally, images can be 

6 provided by/ to the user for space based triggers to help them remember, or even 

7 sort through, the available information. However, both audio and images can 

8 also be stored directly by the user when they press the button on their capture 

9 device. This would help simulate the exact experience that triggered an 

10 individual's desire to commemorate the event to memory. ... hearing a particular 

11 passage in a song or someone making an interesting comment, seeing an image 

12 of a product sitting on a store shelf or an image of meeting attendees for an 

13 important business gathering. . . etc. 

14 Means for A udio Capture 

15 An interesting and novel feature provided by this invention relates to 

16 capturing audio for items of interest. When a user's interest is stimulated by 

17 something they heard (in person or through media), the audio is either 

18 completely done or partially done by the time a user can first decide it is 

19 interesting, and second, presses a record button. The invention disclosed herein 

20 is such that the capture device can be put in a standby mode where a microphone 

21 records all audio, all the time, but retains only some portion of it in a buffer 
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before continuously erasing it. For example, a capture device could always store 
5 or 10 seconds of audio in a recirculating buffer, erasing the oldest audio and 
replacing it with new audio. When a user depresses the button, indicating they 
just heard something of interest, they will be able to freeze the audio buffer and 
then record going forward. For instance, someone could capture the sounds of a 
group assembling for a photo right up to the point when they say "cheese". This 
basic concept of continuous record can also be extended to still or video imaging. 

Downloading Information to the Capture Device 
Although all of the previous scenarios describe a system which captures 
information for downloading to the internet, the invention disclosed herein also 
provides for information and data to flow from the internet to the capture device. 
For all of the scenarios envisioned, the capture device will have a means for 
storing information; this is required for capturing any combination of times, 
venues, audio, images, gps locations... etc. Even the simplest version of the 
capture device will have memory. Therefore, there is an opportunity to provide 
another level of utility to the user and information to the advertisers and 
providers by downloading information to the device. In one of it's simplest 
embodiments, the user, while connected to the dedicated web site would have an 
opportunity to download electronic coupons from various retailers. These 
coupons would be "redeemed" by one of two means: docking their capture 
device at one of the retailer's physical locations or downloading the electronic 
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1 coupon over the internet either as a part of, or separate from, their information 

2 based web site experience. (Additionally, scanned coupons can also be stored 

3 electronically and downloaded by either of the two means.) 

4 User Interaction, Influence and Nielsen ratings 

5 Inherent in the disclosure so far has been the important aspect of 

6 collecting information on users' interests and listening/ watching/ travel patterns 

7 once they have downloaded their placemark information or docked their capture 

8 device in a retail location. This information is not actively given by the user, but 

9 rather taken by the system. Therefore, as far as the user is concerned, 

10 information flows primarily from the internet to them once they have expressed 

11 interest in a particular topic or thing. However, the invention has the novel 

12 feature of allowing users to actively and specifically communicate information to 

13 venue providers and advertisers about their programming - a particular topic, 

14 song, advertisement or question posed with the system described, 

15 Because it is expected that the venues will be actively promoting both the 

16 system and the use of capture devices, programmers, announcers and 

17 commercials will be able to actually prompt users to push a button on their 

18 capture device. They can do this by asking time dependent questions ("All of 

19 you who think a certain candidate should be the next president, press your 

20 button now!" All of you who think a different candidate should be the next 

21 president, press your button now!") or button dependent questions ('Tress your 
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buttons once for candidate #1 and twice for candidate #2!"). Alternatively, 
"yes", "no" or other buttons ("Give me a break", "so-so show".... etc.) could be 
added to (or programmed into) capture devices to get an even more realistic 
dialogue between users and the venue providers. If a fund drive is being held on 
a TV or radio program, the announcer can prompt people to push their button at 
a specific time - that button push can be interpreted by the software differently 
from others so that the very first thing a user may see when he/she connects to 
the web is a screen thanking them for their patronage and asking for a VISA or 
Mastercard number. 

Another provision would be to have standard button patterns established 
within or across venues for users to have a "voice" at all times: 1 quick push 
means "I want more info", 2 quick pushes means "Great program" or "I love this 
song", 1 long push (like leaning on a car horn) could mean "I don't like this song 
at all !" or "I totally disagree with what was just said!". 

Now venue providers and advertisers can get real feedback about how 
their users feel about their programming and users can have real influence about 
what shows up on their TV's and radios. In effect, the system provides a new 
"Nielsen" rating, except that it can encompass a much larger percentage of the 
population, cover many more venues and get specific feedback about program 
content and advertisement, not just determine whether someone has their TV 
tuned to a particular station. When more of the capture devices are web enabled, 
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1 the user feedback to stations will be immediate and not require capture devices 

2 to be docked before user opinions can be determined. 

3 Finally, attention is directed to FIG. 6, which provides a more detailed 

4 consideration of the preferred process flow of the present invention. As shown 

5 therein, when the consumer sees or hears something of interest on the TV or 

6 radio, the user will press a button to capture or placemark the event in memory. 

7 Such button can be connected to a remote control/ PDA, or in that regard, to a 

8 " smart device", as well as a key fob ("non-smart device") or even to a cell phone 

9 or other electronic devices. As a consequence, real time is stored to memory, and 

10 in the case of the "smart device" the device also records venue location, audio, 

11 images and/ or gps coordinates as previously described. These characteristics 

12 can be used individually or in combination to refine the search. 

13 Alternatively, and as illustrated on the right-hand-side of FIG. 6, a 

14 consumer may see printed material (newspaper), and scan the indicia therein 

15 with the capture device of the present invention. (The path for stationary space 

16 based triggers is not specifically shown, but is similar to that for mobile space 

17 based systems. The difference is that gps coordinates are downloaded instead of 

18 indicias.) 

19 Once real time is stored to memory in the capture device of the present 

20 invention, time and venue may be optionally downloaded to a dockable 

21 connection device. Alternatively, the capture device herein may be directly 
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1 connected to a computer via cable or other appropriate means, under those 

2 circumstances where it is not web accessible as a stand-alone product. 

3 As can be seen from FIG, 6, at this point in time the user presses a button 

4 on the device (this can also be programmed to happen automatically). First time 

5 operation leads to an auto/ manual decision point. If manual, the user would 

6 upload the system software, manually, and the capture device would take the 

7 user directly to a dedicated "placemark" venue site ("non-smart" device) or 

8 master "placemark" site in the case of the "smart" device. Times, venues, indicia 

9 ID's, audio, video, gps coordinates and user/ device ID would be downloaded to 

10 such sites, and a look-up table would define viewer displayed information based 

11 on earlier mentioned considerations: venue, program, section, time (or location), 

12 description, sponsors, etc. Finally, clicking on the description of any item (or 

13 designated screen button) takes the user directly to the URL, supplied by the 

14 venue provider/ programmer, for the associated times, indicias or coordinates. 

15 The master "placemark" site would be built in part from information supplied by 

16 each of the dedicated "placemark" sites. 

17 As also illustrated in FIG. 6, in that situation wherein the device is being 

18 used for the first time, after placemarking by the user, the device may itself 

19 automatically provide that software is loaded from a website, or from the device 

20 itself, as opposed to having the user load software manually. The device would 
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1 then proceed in the manner outlined above with regard to delivering the user to 

2 a page. 

3 Finally, as noted above, in that situation where it is not the first time use 

4 of the device, after placemarking by the user, the device would take the user 

5 directly to either the page of a dedicated "placeinark" venue ("non-smart") or 

6 master "placemark" site ('smart"). That is, there would be no need for software 

7 loading, as previously described, and the remaining processing options would be 

8 identical 

9 Those skilled in the art will therefore appreciate that numerous 

10 alternatives to the invention as described herein are possible, without departing 

11 from the spirit and scope of the disclosure herein. 
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